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An analogue of the mixing property of quantum entropy is derived for quantum relative entropy. 
It is applied to the final state of ideal measurement and to the spectral form of the second density 
operator. Three cases of states on a directed straight line of relative entropy are discussed. 
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Relative entropy plays a fundamental role in quantum 
information theory (see p. 15 in ^ and the review arti- 
cles Q, which have relative entropy in the title). 

The relative entropy S{p\\a) of a state (density opera- 
tor) p with respect to a state a is by definition 

S{p\\a) =tr[plog{p)] - tr[plog{(T)] (la) 



if supp(p) C supp(cr); (lb) 

or else S'(p||(t) = -l-oo (Ic) 

(see p. 16 in 1]). By "support" is meant the subspace 
that is the topological closure of the range. 

If a is singular and condition (lb) is valid, then the 
orthocomplement of the support (i. e., the null space) 
of p, contains the null space of a, and both operators 
reduce in supp((T). Relation (lb) is valid in this subspace. 
Both density operators reduce also in the null space of a. 
Here the log is not defined, but it comes after zero, and 
it is generally understood that zero times an undefined 
quantity is zero. We'll refer to this as the zero convention. 

The more familiar concept of (von Neumann) quantum 
entropy, S{p) = —tT[plog{p)], also requires the zero con- 
vention. If the state space is infinite dimensional, then, 
in a sense, entropy is almost always infinite (cf p. 241 in 
0). In finite-dimensional spaces, entropy is always finite. 

In contrast, relative entropy is often infinite also in 
finite-dimensional spaces (due to (Ic)). Most results on 
relative entropy with general validity are inequalities, and 
the infinity fits well in them. It is similar with entropy. 
But there is one equality for entropy that is much used, 
the mixing property concerning orthogonal state decom- 
position (cf p. 242 in 3): 



H{wk) = ~y^}wklog{wk)] 



(36) 



Wh<Jk, 



(2) 



Vfc : Wk> 0; for Wk > 0, ak> 0, tro-fc = 1; Y,k '^k = 1- 
Then 



S{a-) = H{wk) + WkS{(Tk), 



(3a) 



being the Shannon entropy of the probability distribution 
{wk ■■ Vfe}. 

The first aim of this article is to derive an analogue 
of (3a), which will be called mixing property of relative 
entropy. The second aim is to apply it to the derivation 
of two properties of the final state in ideal measurement, 
and to the spectral decomposition of a in the general 
case. 

We will find it convenient to make use of an extension 
log'^ of the logarithmic function to the entire real axis: 

if < a; : log^ix) = log{x), (4a) 



if X < : log^'ix) = 0. (46) 

The following elementary property of the extended log- 
arithm will be utilized. 

Lemma 1: If an orthogonal state decomposition (2) is 
given, then 

lo9%^) = J2^log{wk)]Qk + J2 '«.9'K)' (5) 

k k 

where Qk is the projector onto the support of ak, and the 
prim on the sum means that the terms corresponding to 
Wk ~ are omitted. 

Proof: Spectral forms Vfc, > : Ok — 

J2ik'^ik I h){lk I (all si^ positive) give a spectral 
form cr = J2kJ2i^WkSi^ \ lk){lk \ of cr on account of 
the orthogonality assumed in (2) and the zero conven- 
tion. Since numerical functions define the corresponding 
operator functions via spectral forms, one obtains further 

loggia) = ^^[?og'=(u;fcSiJ] \ lk){lk\ = 



^^[^og{wk) + log{si^)] \ lk){lk\ 

k Ik 
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Y,[log{wk)]Qk + J2J2^log{siJ] \lk){lk 



k h. 
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(In the last step Qk = Y^i^ I lk){h \ for Wk> Q was made 
use of.) The same is obtained from the RHS of (5) when 
the spectral forms of ak are substituted in it. □ 
Now we come to the main result. 

Theorem 1: Let condition (lb) be valid for the states p 
and a, and let an orthogonal state decomposition (2) be 
given. Then 

S{p\\a) = s{j2QkpQ^)~S{p)^ 



H{pk\\wk) + y^^PkS{QkpQk/Pk\\crk), 



(6) 



where, for Wk > 0, Qk projects onto the support of ak, 
and Qk = if Wk = 0, pk = tr{pQk), and 

H{pk\\wk) = ^[pklog{pk)] - ^[pklog{wk)] (7) 



(Note that Wk > because > by assumption.) Thus, 
Qk \ ipn) ^ Qk \ (j^i) & supp((Tfe). This is valid for 
any nonzero vector appearing in (10b), and these span 
supp(pfc) (cf Appendix (ii)). Therefore, (9) is valid. 

On account of (lb), the standard logarithm can be 
replaced by the extended one in definition (la) of relative 
entropy: 

Sip\\cT) = -S{p)-tT[plog^a)]. 

Substituting (2) on the RHS, and utilizing (5), the rela- 
tive entropy S'(/3||(t) becomes 

^S{p) - tT[p[j2[log{wk)]Qk + J2^log''^ak)]] } - 



'S{p) -'^[Pklog{wk)] -^trlplog^iak)]- 



is the classical discrete counterpart of the quantum rela- 
tive entropy, valid because {pk > 0) ^ {wk > 0). 

One should note that the claimed validity of the classi- 
cal analogue of (lb) is due to the definitions of and Qk- 
Besides, (2) implies that {J2k Qk) projects onto supp(tT). 
Further, as a consequence of (lb), (^j, Qk)p = P- Hence, 

tr[EkQkpQk) = 1. 

Proof of theorem 1 : We define 



}k/Pk 



(8) 



Wk, Pk > ■■ Pk = Qk 

First we prove that (lb) implies 

V/c, Pk > ■■ supp(pfe) C supp(o-/c)- (9) 

Let k, Pk > 0, be an arbitrary fixed value. We take a 
pure-state decomposition 



(10a), 



Vn : A„ > 0. Applying Qk---Qk to (10a), one obtains 
another pure-state decomposition 



QkPQk = PkPk = ^ AnQfc | Qk 



(105) 



(cf (8)). Let Qk I Vn) be a nonzero vector appearing 
in (10b). Since (10a) imphes that | Vn) S supp(p) (cf 
Appendix (ii)), condition (lb) further implies | Vn) G 
supp(cr). Let us write down a pure-state decomposition 



)(0n 



(11) 



with I (f)i) =1 Vn)- (This can be done with A'^ > cf 0.) 
Then, applying Qk---Qk to (11) and taking into account 
(2), we obtain the pure-state decomposition 

QkO-Qk = WkCTk = ^ KnQk |'^m)(0m | Qk- (H^) 



Adding and subtracting H{pk) (cf (3b)), replacing 
log'^[(7k) by Qk\log^{ak)\Qk, and taking into account (7) 
and (8), one further obtains 

S{p\\a) = ~S{p) -f H{pk) -f H{pk\\wk)+ 



-^Pk^'^lPklog'^iak)]- 

k 

(The zero convention is valid for the last term because 
the density operator QkpQk/Pk may not be defined. Note 
that replacing J2k Yl'k C^) *1°^^ ^'^^ change the LHS 
because only pk = terms are omitted.) 

Adding and subtracting the entropies S{pk) in the sum, 
one further has 

S{p\\a) = -S{p) -f H{pk) -f H{pk\\wk) + 



^PkS{pk) + '^Pk{-S{pk) - tr[pklog^{ak)]}- 

k k 

Utilizing the mixing property of entropy (3a), one can 
put s(^J2kPkPk^ instead of [H{pk) + J2'kPkS{pk)]- 

Owing to (9), we can replace log"^ by the standard 
logarithm and thus obtain the RHS (6). □ 

Some Applications of the Mixing Property - Let p be 
a state and A = J^i '^i^i + ^jPj ^ spectral form of 
a discrete observable (Hermitian operator) A, where the 
eigenvalues Oi and aj are all distinct. The index i enumer- 
ates all the detectable eigenvalues, i. e., Vi : tr{pPi) > 0, 

andtr[p(E,P.)] = l. 

After an ideal measurement of A in p, the entire en- 
semble is described by the Liiders state: 



PLiA) = J2P^pPi 



(12) 
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(cf (One can take moregeneral observables that are 
ideally measurable in p cf Q- For simplicity we confine 
ourselves to discrete ones.) 

Corollary 1: The relative- entropic " distance" from any 
quantum state to its Liiders state is the difference between 
the corresponding quantum entropies: 



i i 

Proof: First we must prove that 



supp(p) C suppf ^f^pf, 



(14) 



To this purpose, we write down a decomposition (10a) 
of p into pure states. One has supp(^-Pi) 3 supp(p) 
(equivalent to the certainty of (X^i^O Pi cf |3l)j and 
the decomposition (10a) implies that each | tjiri) belongs 
to supp(/9). Hence, | ■!/)„) S supp(^jPi); equivalently, 
I V'n) — (X^i Pi) I V'n)- Therefore, one can write 

Vn: =5](^. iV-n)). (15a) 

i 

Further, (10a) implies 

i in 

As seen from (15b), all vectors {Pi \ ipn )) belong to 
supp(X]i ^iP-Pi)- Hence, so do all | -^n) (due to (15a)). 
Since p is the mixture (10a) of the IV'n), the latter span 
supp(p). Thus, finally, also (14) follows. 

In our case cr = J^iPiPPi (6)- We replace k by i. 
Next, we establish 



Vi : QipQi = PipPi- 



(16) 



Since Qi is, by definition, the support projector of 
(P.pPO, and P^iP^pP^) = iP^pP^), one has P,Q, = Q, 
(see Appendix (i)). One can write PipPi = Qi{PipPi)Qi, 
from which then (16) follows. 

Realizing that Wi = tT:(QipQi) = tr{PipPi) = pi due 
to (16), one obtains H{pi\\'Wi) = and 

Vz: S{Q,pQ,/p,\\P,pP,lw,)=Q 

in (6) for the case at issue. This completes the proof. □ 
Now we turn to a peculiar further implication of corol- 
lary 1. 

Let B = X^fcX^ifc bkii^Pkik be a spectral form of a dis- 
crete observable (Hermitian operator) B such that all 
eigenvalues bki^. are distinct. Besides, let B be more com- 
plete than A or, synonymously, a refinement of the latter. 
This, by definition means that 



Vfc 



Pk 



y^^Pkik 



(17) 



is valid. Here k enumerates both the i and the j index 
values in the spectral form of A. 

Let Pl{A) and pl{B) be the Liiders states (12) of p 
with respect to A and B respectively. 

Corollary 2: The states p, pl(A), and Pl{B) lie on a 
straight line with respect to relative entropy, i. e. 

s(p\\pl{B)) ^ s(p\\pl{A)) + s(pl{A))\\pl{B)), 

(18a) 

or explicitly: 

S[P\\Y.Y.^P^UPP^U)) = S[p\\Y,{PrPP.) 

i h i 



s[Y.^P.PP^)\\Y,Y.^Pa.pPri 



i L 



(186) 



Note that all eigenvalues bki^ of B with indices others 
than Hi arc undetectable in p. 

Proof follows immediately from corollary 1 because 

s{p\\pl{B)) - [s[pLiB))^s[pLiA)) 



s 



[pl{A))-S{p) 



and, as easily seen from (12), pl{B) = (^pl{A)^ (B) due 

to Pu,P^' =Su,Pu, (cf (17)). ^ □ 

Next, we derive another consequence of theorem 1. 
Corollary 3: Let {pk : Vfc} and {wk '■ Vfc} be probability 

distributions such that pk > Wk> 0- Then, 

H{p,\\wk) = s(^J2P'' llE^'= \k){k\), (19) 

k k 

where the LHS is given by (7), and the orthonormal set 
of vectors {| fc) : Vfc} is arbitrary. 

Proof: Applying (6) to the RHS of (19), one obtains 



i?i/5(19) ^ s(Y.pk\k){k\) - s(Y.pk |fc)(fc 



iJ(pfc||«;fc)-H5^Pfc5(|fc)(fc| II \k){k\) = LHS{l^). 



□ 



Finally, a quite different general result also follows from 
the mixing property (6). 

Theorem 2: Let S{p\\a) be the relative entropy of any 
two states such that (lb) is satisfied. Let, further. 



cr = ^ Wfc I fc}(fc I 



(20) 



be a spectral form of a in terms of eigenvectors. Then 

s{p\\<7)^s(p\\Y,m{k\p\k){k\ 
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S[j2{\k){k\p\k){k\)\\a). (21) 

A: 

Thus, the states p, Y.ki\ ^)(^ I P I ^)(^ I) ('c/ (20) for 
k){k \), and a lie on a directed straight line of relative 
entropy. 

Proof: Application of (6) to the LHS(21), in view of 
(20), leads to 

S{p\\a)^s[j2{\k){k\p\k){k\))-S{p)+ 

k 

Hipk\\wk)+Y,PkS{\k){k\ II |fc)(fc|). 

k 

In view of Pk = {k \ p \ k), (13), (19), and (20), this 
equals RHS(21). □ 
It is well known that the relative-entropic " distance" , 
unlike the Hilbert-Schmidt (HS) one, fails to satisfy the 
triangle rule, which requires that the distance between 
two states must not exceed the sum of distances if a third 
state is interpolated. But, and this is part of the triangle 
rule, one has equality if and only if the interpolated state 
lies on a straight line with the two states. As it is seen 
from corollary 2 and theorem 2 as examples, the relative- 
entropic " distance" does satisfy the equality part of the 
triangle rule. 

An interpolated state lies on the HS line between two 
states if and only if it is a convex combination of the 
latter. Evidently, this is not true in the case of relative 
entropy. 

Partovi @ has recently considered three states on a di- 
rected relative-entropic line: a general multipartite state 
pi = PAB...N, a suitable separable multipartite state 
P2 = p^Q ^ with the same reductions pA, Pb, ■ ■ ■ , PN: 
and finally ps = pa® Pb (E) ■ ■ ■ (Ei Pn- The mutual infor- 
mation in pi is taken to be its total correlations infor- 
mation. It is well known that it can be written as the 
relative entropy of pi relative to ps. The straight line 
implies: 

S{pi\\p2) = 5'(pi||p3) - S'(p2||P3)- 

To my understanding, it is Partovi's idea that if p2 is as 
close to pi as possible (but being on the straight line and 



having the same reductions), then its von Neumann mu- 
tual information S{p2\\p3) equals the classical informa- 
tion in pi, and S'(pi||p2) is the amount of entanglement 
or quantum correlation information in pi . 

Partovi's approach utilizes the relative-entropy "dis- 
tance" in the only way how it is a distance: on a straight 
line. One wonders why should the relative entropy "dis- 
tance" be relevant outside a straight line, where it is no 
distance at all cf 0- On the other hand, these ap- 
proaches have the very desirable property of being enten- 
glement monotones. But so are many others (see ibid.). 

To sum up, we have derived the mixing property of 
relative entropy S'(p||cr) for the case when (lb) is valid 
(theorem 1), and two more general equalities of relative 
entropies (corollary 3 and theorem 2), which follow 
from it. Besides, two properties of Liiders states (12) 
have been obtained (corollary 1 and corollary 2). The 
mixing property is applicable to any orthogonal state 
decomposition (2) of a. Hence, one can expect a ver- 
satility of its applications in quantum information theory. 

Appendix - Let p = '^^^Xn \ n){n \ be an arbitrary 
decomposition of a density operator into ray projectors, 
and let E be any projector. Then 

Ep^p ^ \fn: E\n)=\n) {A.l) 

(cf Lemma A.l. and A. 2. in [ill)- 

(i) If the above decomposition is an eigendecomposi- 
tion with positive weights, then |n)(n|= Q, Q being 
now the support projector of p, and, on account of (A.l), 

Ep^ P =^ EQ::=Q. {A.2) 



(ii) Since one can always write Qp = p, (A.l) implies 
that all I n) in the arbitrary decomposition belong 
to supp(p). Further, defining a projector F so that 
supp(F) = span({| n) : Vn}), one has FQ = F. Equiva- 
lence (A.l) imphes Fp = p. Hence, (A.2) gives QF = Q. 
Altogether, F — Q, i. e., the unit vectors {\ n) : Vn} 
span supp(p). 
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